Plots:

Full plots - For the full plots, Each plot represents a unique simulation and the values in the cells are the mean value across all of the 10 iterations of that simulation across all three unique landscape seeds (i.e., all three sets of Neutral Landscape Models) for a total of 30 replicates. Sample strategy is on the y-axis and number of sites is on the x-axis. Since there are a many simulations presented in the full plots, here is a handy key for how the different parameters are laid out within them (H = High, L = Low):

TPR - True Positive Rate. The proportion of times there was a positive detection in both the sub-sampled model and the full model. Note that in the case of GDM TPR is frequently NA because there was no detection in the full model.

FDR - False Discovery Rate. The proportion of times there was a detection in the sub-sampled model that was not shared with the full model.

ME - mean error was calculated by taking the mean difference between the observed and expected coefficients, because the absolute value is not taken this measurement is used to determine whether over- or underestimation is occurring. For IBE ME the two environmental coefficient errors were averaged.

MAE - mean absolute error calculated as the difference between the observed and expected coefficients. For IBE MAE the two environmental coefficient errors were averaged.

Proportion of negative & significant coefficients - this only applies to MMRR and IBE and is the proportion of coefficients that were negative & significant.

Proportion NA - this only applies to GDM and IBE and is the proportion of NA p-values due to the variable significance procedure note being able to be carried out because more than two variable coefficients in the model were zero or because one of the models used in the calculation could not be fit. Note that p-values would also be NA in the case of the variable coefficient was zero, but we treated these as cases of no detection in our calculation of TPR/FDR, whereas the other NA values were excluded.

1. MMRR

1.1 Individual sampling

1.1.1 Linear mixed effects models

There is no model table for IBD TPR because all values are fixed or NA There is no model table for IBD FDR because all values are fixed or NA
Linear mixed effect model
IBD MAE ~ sample number + sampling strategy + population size + migration + selection strength + spatial autocorrelation + environmental correlation + (1 | seed)
Predictors Fixed Effects Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
Sample number −0.0001 0.9573 0.9573 1 15.3480K 1173.276938 7.5 × 10−248*
Population size 0.0049 0.0934 0.0934 1 15.3480K 114.477840 1.3 × 10−26*
Migration 0.0254 2.4797 2.4797 1 15.3480K 3039.258164 0.0*
Selection strength −0.0009 0.0032 0.0032 1 15.3480K 3.930859 0.047**
Spatial autocorrelation 0.0051 0.0998 0.0998 1 15.3480K 122.274384 2.6 × 10−28*
Environmental correlation −0.0009 0.0033 0.0033 1 15.3480K 4.013043 0.045**
* p < 0.001
** p < 0.05
Tukey test for IBD MAE
pairwise ~ sampstrat
Contrast Estimate SE Z ratio p p
ES - G −0.0014 0.0007 −2.1046 0.1515553 0.1500
ES - R −0.0023 0.0007 −3.4765 2.9 × 10−3 0.0029
ES - T −0.0234 0.0007 −35.8568 0.0 0.0000
G - R −0.0009 0.0007 −1.3719 0.5170770 0.5200
G - T −0.0220 0.0007 −33.7521 0.0 0.0000
R - T −0.0211 0.0007 −32.3803 0.0 0.0000
Linear mixed effect model
IBE TPR ~ sample number + sampling strategy + population size + migration + selection strength + spatial autocorrelation + environmental correlation + (1 | seed)
Predictors Fixed Effects Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
Sample number 0.0017 168.7131 168.7131 1 11.9880K 1373.80433 8.5 × 10−285*
Population size −0.1081 34.9283 34.9283 1 11.9880K 284.41535 4.4 × 10−63*
Migration −0.4291 482.7370 482.7370 1 11.9881K 3930.85191 0.0*
Selection strength 0.0687 14.0422 14.0422 1 11.9881K 114.34357 1.4 × 10−26*
Spatial autocorrelation 0.2713 195.0765 195.0765 1 11.9880K 1588.47764 0.0*
Environmental correlation −0.0554 9.1634 9.1634 1 11.9882K 74.61613 6.4 × 10−18*
* p < 0.001
Tukey test for IBE TPR
pairwise ~ sampstrat
Contrast Estimate SE Z ratio p p
ES - G 0.0947 0.0090 10.4624 3.8 × 10−14 3.8e-14
ES - R 0.0778 0.0090 8.6020 3.6 × 10−14 3.6e-14
ES - T −0.0217 0.0090 −2.3946 0.0780902 7.8e-02
G - R −0.0168 0.0090 −1.8604 0.2452988 2.5e-01
G - T −0.1163 0.0090 −12.8569 0.0 0.0e+00
R - T −0.0995 0.0090 −10.9966 3.0 × 10−14 3.0e-14
Linear mixed effect model
IBE FDR ~ sample number + sampling strategy + population size + migration + selection strength + spatial autocorrelation + environmental correlation + (1 | seed)
Predictors Fixed Effects Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
Sample number 0.0001 1.1762 1.1762 1 15.3480K 50.23842 1.4 × 10−12*
Population size 0.0138 0.7315 0.7315 1 15.3480K 31.24393 2.3 × 10−8*
Migration 0.0115 0.5042 0.5042 1 15.3480K 21.53373 3.5 × 10−6*
Selection strength −0.0173 1.1516 1.1516 1 15.3480K 49.18778 2.4 × 10−12*
Spatial autocorrelation −0.0491 9.2532 9.2532 1 15.3480K 395.21796 7.5 × 10−87*
Environmental correlation 0.0137 0.7178 0.7178 1 15.3480K 30.65721 3.1 × 10−8*
* p < 0.001
Tukey test for IBE FDR
pairwise ~ sampstrat
Contrast Estimate SE Z ratio p p
ES - G 0.0074 0.0035 2.1254 0.14500630 1.5e-01
ES - R 0.0108 0.0035 3.0949 0.01061518 1.1e-02
ES - T −0.0253 0.0035 −7.2338 2.8 × 10−12 2.8e-12
G - R 0.0034 0.0035 0.9695 0.76685022 7.7e-01
G - T −0.0327 0.0035 −9.3591 3.9 × 10−14 3.9e-14
R - T −0.0361 0.0035 −10.3286 3.9 × 10−14 3.9e-14
Linear mixed effect model
IBE MAE ~ sample number + sampling strategy + population size + migration + selection strength + spatial autocorrelation + environmental correlation + (1 | seed)
Predictors Fixed Effects Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
Sample number −0.0002 1.7981 1.7981 1 15.3480K 3352.290944 0.0*
Population size 0.0020 0.0150 0.0150 1 15.3480K 27.904871 1.3 × 10−7*
Migration 0.0211 1.7070 1.7070 1 15.3480K 3182.358194 0.0*
Selection strength −0.0015 0.0089 0.0089 1 15.3480K 16.661289 4.5 × 10−5*
Spatial autocorrelation 0.0005 0.0010 0.0010 1 15.3480K 1.793178 0.18
Environmental correlation 0.0004 0.0006 0.0006 1 15.3480K 1.195895 0.27
* p < 0.001
Tukey test for IBE MAE
pairwise ~ sampstrat
Contrast Estimate SE Z ratio p p
ES - G 0.0003 0.0005 0.5086 0.9570719 0.96
ES - R 0.0012 0.0005 2.2604 0.1074386 0.11
ES - T −0.0071 0.0005 −13.4163 0.0 0.00
G - R 0.0009 0.0005 1.7518 0.2969312 0.30
G - T −0.0074 0.0005 −13.9249 0.0 0.00
R - T −0.0083 0.0005 −15.6767 0.0 0.00

1.1.2 Full plots

1.2 Site sampling

1.2.1 Linear mixed effects models

Linear mixed effect model
IBD TPR ~ sample number + sampling strategy + population size + migration + selection strength + spatial autocorrelation + environmental correlation + (1 | seed)
Predictors Fixed Effects Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
Sample number 0.0002 0.0128 0.0128 1 8.6290K 12.392602 4.3 × 10−4*
Population size −0.0016 0.0057 0.0057 1 8.6290K 5.470225 0.019**
Migration −0.0021 0.0094 0.0094 1 8.6290K 9.042617 2.6 × 10−3***
Selection strength −0.0016 0.0057 0.0057 1 8.6290K 5.470225 0.019**
Spatial autocorrelation −0.0007 0.0010 0.0010 1 8.6290K 1.004735 0.320
Environmental correlation 0.0016 0.0057 0.0057 1 8.6290K 5.470225 0.019**
* p < 0.001
** p < 0.05
*** p < 0.01
Tukey test for IBD TPR
pairwise ~ sampstrat
Contrast Estimate SE Z ratio p p
ES - G 0.0000 0.0008 0.0000 1.0000000 1.00
ES - R 0.0010 0.0008 1.2276 0.4367856 0.44
G - R 0.0010 0.0008 1.2276 0.4367856 0.44
There is no model table for IBD FDR because all values are fixed or NA
Linear mixed effect model
IBD MAE ~ sample number + sampling strategy + population size + migration + selection strength + spatial autocorrelation + environmental correlation + (1 | seed)
Predictors Fixed Effects Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
Sample number −0.0008 0.2436 0.2436 1 8.6290K 1.233529e+02 1.8 × 10−28*
Population size 0.0205 0.9093 0.9093 1 8.6290K 4.603368e+02 1.6 × 10−99*
Migration 0.1257 34.1279 34.1279 1 8.6290K 1.727823e+04 0.0*
Selection strength −0.0080 0.1400 0.1400 1 8.6290K 7.086196e+01 4.4 × 10−17*
Spatial autocorrelation 0.0018 0.0074 0.0074 1 8.6290K 3.729109e+00 0.054
Environmental correlation −0.0001 0.0000 0.0000 1 8.6290K 1.279521e-02 0.910
* p < 0.001
Tukey test for IBD MAE
pairwise ~ sampstrat
Contrast Estimate SE Z ratio p p
ES - G −0.0061 0.0012 −5.2174 5.4 × 10−7 5.4e-07
ES - R −0.0074 0.0012 −6.3517 6.4 × 10−10 6.4e-10
G - R −0.0013 0.0012 −1.1343 0.4929179 4.9e-01
Linear mixed effect model
IBE TPR ~ sample number + sampling strategy + population size + migration + selection strength + spatial autocorrelation + environmental correlation + (1 | seed)
Predictors Fixed Effects Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
Sample number 0.0098 27.7363 27.7363 1 6.7390K 306.147266 4.5 × 10−67*
Population size −0.0160 0.4297 0.4297 1 6.7390K 4.743160 0.029**
Migration −0.0303 1.3530 1.3530 1 6.7393K 14.933732 1.1 × 10−4*
Selection strength 0.0213 0.7591 0.7591 1 6.7393K 8.378417 3.8 × 10−3***
Spatial autocorrelation 0.1005 15.0498 15.0498 1 6.7391K 166.116605 1.4 × 10−37*
Environmental correlation 0.0087 0.1283 0.1283 1 6.7395K 1.416395 0.230
* p < 0.001
** p < 0.05
*** p < 0.01
Tukey test for IBE TPR
pairwise ~ sampstrat
Contrast Estimate SE Z ratio p p
ES - G 0.0878 0.0090 9.7814 3.2 × 10−14 3.2e-14
ES - R 0.0627 0.0090 6.9832 8.7 × 10−12 8.7e-12
G - R −0.0251 0.0090 −2.7982 0.01422545 1.4e-02
Linear mixed effect model
IBE FDR ~ sample number + sampling strategy + population size + migration + selection strength + spatial autocorrelation + environmental correlation + (1 | seed)
Predictors Fixed Effects Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
Sample number 0.0010 0.3605 0.3605 1 8.6290K 21.53339 3.5 × 10−6*
Population size 0.0105 0.2396 0.2396 1 8.6290K 14.31216 1.6 × 10−4*
Migration 0.0332 2.3834 2.3834 1 8.6290K 142.35944 1.5 × 10−32*
Selection strength −0.0122 0.3190 0.3190 1 8.6290K 19.05466 1.3 × 10−5*
Spatial autocorrelation −0.0367 2.9077 2.9077 1 8.6290K 173.67648 2.8 × 10−39*
Environmental correlation 0.0108 0.2503 0.2503 1 8.6290K 14.94818 1.1 × 10−4*
* p < 0.001
Tukey test for IBE FDR
pairwise ~ sampstrat
Contrast Estimate SE Z ratio p p
ES - G 0.0101 0.0034 2.9531 8.8 × 10−3 0.0088
ES - R 0.0056 0.0034 1.6293 0.2332438 0.2300
G - R −0.0045 0.0034 −1.3238 0.3818568 0.3800
Linear mixed effect model
IBE MAE ~ sample number + sampling strategy + population size + migration + selection strength + spatial autocorrelation + environmental correlation + (1 | seed)
Predictors Fixed Effects Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
Sample number −0.0023 1.9910 1.9910 1 8.6290K 1332.950051 1.6 × 10−271*
Population size 0.0028 0.0168 0.0168 1 8.6290K 11.261671 7.9 × 10−4*
Migration 0.0266 1.5251 1.5251 1 8.6290K 1021.052826 7.4 × 10−212*
Selection strength −0.0016 0.0054 0.0054 1 8.6290K 3.611645 0.057
Spatial autocorrelation −0.0008 0.0015 0.0015 1 8.6290K 1.037414 0.310
Environmental correlation 0.0054 0.0631 0.0631 1 8.6290K 42.266725 8.4 × 10−11*
* p < 0.001
Tukey test for IBE MAE
pairwise ~ sampstrat
Contrast Estimate SE Z ratio p p
ES - G 0.0006 0.0010 0.5942 0.82324147 0.8200
ES - R 0.0031 0.0010 3.0406 6.7 × 10−3 0.0067
G - R 0.0025 0.0010 2.4464 0.03834878 0.0380

1.2.2 Full plots

2. GDM

2.1 Individual sampling

2.1.1 Linear mixed effects models

There is no model table for IBD TPR because all values are fixed or NA There is no model table for IBD FDR because all values are fixed or NA
Linear mixed effect model
IBD MAE ~ sample number + sampling strategy + population size + migration + selection strength + spatial autocorrelation + environmental correlation + (1 | seed)
Predictors Fixed Effects Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
Sample number −0.0002 4.0044 4.0044 1 15.3410K 2411.691502 0.0*
Population size 0.0035 0.0469 0.0469 1 15.3410K 28.231770 1.1 × 10−7*
Migration −0.0178 1.2169 1.2169 1 15.3410K 732.874516 1.1 × 10−157*
Selection strength 0.0022 0.0187 0.0187 1 15.3410K 11.285347 7.8 × 10−4*
Spatial autocorrelation 0.0024 0.0228 0.0228 1 15.3410K 13.752010 2.1 × 10−4*
Environmental correlation 0.0013 0.0062 0.0062 1 15.3410K 3.743701 0.053
* p < 0.001
Tukey test for IBD MAE
pairwise ~ sampstrat
Contrast Estimate SE Z ratio p p
ES - G 0.0146 0.0009 15.6623 0.0 0.0e+00
ES - R 0.0027 0.0009 2.8747 0.02107935 2.1e-02
ES - T 0.0086 0.0009 9.2769 4.3 × 10−14 4.3e-14
G - R −0.0119 0.0009 −12.7881 0.0 0.0e+00
G - T −0.0059 0.0009 −6.3892 1.0 × 10−9 1.0e-09
R - T 0.0060 0.0009 6.4018 9.2 × 10−10 9.2e-10
Linear mixed effect model
IBE TPR ~ sample number + sampling strategy + population size + migration + selection strength + spatial autocorrelation + environmental correlation + (1 | seed)
Predictors Fixed Effects Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
Sample number 0.0018 26.6321 26.6321 1 1.6320K 380.475372 2.5 × 10−76*
Population size −0.1837 7.3015 7.3015 1 1.6332K 104.311892 8.8 × 10−24*
Migration −0.2489 4.1555 4.1555 1 1.6339K 59.366632 2.3 × 10−14*
Selection strength 0.0728 1.8054 1.8054 1 1.6340K 25.792338 4.2 × 10−7*
Spatial autocorrelation 0.1231 3.3848 3.3848 1 1.6339K 48.356558 5.1 × 10−12*
Environmental correlation −0.0051 0.0075 0.0075 1 1.6329K 0.107275 0.74
* p < 0.001
Tukey test for IBE TPR
pairwise ~ sampstrat
Contrast Estimate SE t ratio p p
ES - G 0.0549 0.0185 2.9773 0.0156144 1.6e-02
ES - R 0.0148 0.0185 0.8016 0.8536761 8.5e-01
ES - T −0.0787 0.0185 −4.2628 1.3 × 10−4 1.3e-04
G - R −0.0402 0.0185 −2.1757 0.1304521 1.3e-01
G - T −0.1336 0.0185 −7.2402 3.4 × 10−12 3.4e-12
R - T −0.0935 0.0185 −5.0644 2.7 × 10−6 2.7e-06
Linear mixed effect model
IBE FDR ~ sample number + sampling strategy + population size + migration + selection strength + spatial autocorrelation + environmental correlation + (1 | seed)
Predictors Fixed Effects Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
Sample number 0.0001 0.3650 0.3650 1 14.7952K 52.094999 5.5 × 10−13*
Population size 0.0047 0.0811 0.0811 1 14.7953K 11.582487 6.7 × 10−4*
Migration −0.0014 0.0075 0.0075 1 14.7952K 1.071583 0.3
Selection strength 0.0018 0.0117 0.0117 1 14.7953K 1.667097 0.2
Spatial autocorrelation 0.0037 0.0497 0.0497 1 14.7957K 7.098618 7.7 × 10−3**
Environmental correlation −0.0037 0.0496 0.0496 1 14.7962K 7.081235 7.8 × 10−3**
* p < 0.001
** p < 0.01
Tukey test for IBE FDR
pairwise ~ sampstrat
Contrast Estimate SE Z ratio p p
ES - G 0.0005 0.0020 0.2370 0.9953179 1.0e+00
ES - R −0.0011 0.0019 −0.5444 0.9480834 9.5e-01
ES - T −0.0136 0.0019 −6.9998 1.5 × 10−11 1.5e-11
G - R −0.0015 0.0020 −0.7804 0.8633702 8.6e-01
G - T −0.0140 0.0019 −7.2237 3.1 × 10−12 3.1e-12
R - T −0.0125 0.0019 −6.4570 6.4 × 10−10 6.4e-10
Linear mixed effect model
IBE MAE ~ sample number + sampling strategy + population size + migration + selection strength + spatial autocorrelation + environmental correlation + (1 | seed)
Predictors Fixed Effects Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
Sample number −0.0001 0.2034 0.2034 1 15.3390K 1896.2107557 0.0*
Population size −0.0011 0.0050 0.0050 1 15.3390K 47.0447413 7.2 × 10−12*
Migration 0.0031 0.0375 0.0375 1 15.3390K 349.3449763 4.2 × 10−77*
Selection strength 0.0005 0.0008 0.0008 1 15.3390K 7.6256344 5.8 × 10−3**
Spatial autocorrelation −0.0002 0.0001 0.0001 1 15.3390K 0.8190549 0.37
Environmental correlation −0.0008 0.0022 0.0022 1 15.3390K 20.7301418 5.3 × 10−6*
* p < 0.001
** p < 0.01
Tukey test for IBE MAE
pairwise ~ sampstrat
Contrast Estimate SE Z ratio p p
ES - G 0.0002 0.0002 0.9207 0.7937629 7.9e-01
ES - R −0.0002 0.0002 −1.0063 0.7457560 7.5e-01
ES - T −0.0026 0.0002 −10.8855 3.6 × 10−14 3.6e-14
G - R −0.0005 0.0002 −1.9268 0.2167225 2.2e-01
G - T −0.0028 0.0002 −11.8042 0.0 0.0e+00
R - T −0.0023 0.0002 −9.8791 3.8 × 10−14 3.8e-14

2.1.2 Full plots

2.1.4 Failed fits

Occasionally GDM fails to fit a model, in which case an NA value is assigned. Here we check the proportion of NAs (i.e., cases of failed fit) across the simulations:

Proportion of failed full models:

2.2 Site sampling

2.2.1 Linear mixed effects models

Linear mixed effect model
IBD TPR ~ sample number + sampling strategy + population size + migration + selection strength + spatial autocorrelation + environmental correlation + (1 | seed)
Predictors Fixed Effects Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
Sample number 0.0005 0.0732 0.0732 1 7.7683K 28.70723745 8.7 × 10−8*
Population size −0.0020 0.0078 0.0078 1 7.7682K 3.07132080 0.080
Migration −0.0047 0.0424 0.0424 1 7.7681K 16.63656418 4.6 × 10−5*
Selection strength 0.0001 0.0000 0.0000 1 7.7693K 0.01226572 0.910
Spatial autocorrelation −0.0021 0.0082 0.0082 1 7.7682K 3.21805214 0.073
Environmental correlation −0.0007 0.0010 0.0010 1 7.7687K 0.40331611 0.530
* p < 0.001
Tukey test for IBD TPR
pairwise ~ sampstrat
Contrast Estimate SE Z ratio p p
ES - G −0.0017 0.0014 −1.2259 0.4378351 0.44
ES - R −0.0012 0.0014 −0.8235 0.6883972 0.69
G - R 0.0005 0.0014 0.3931 0.9183676 0.92
There is no model table for IBD FDR because all values are fixed or NA
Linear mixed effect model
IBD MAE ~ sample number + sampling strategy + population size + migration + selection strength + spatial autocorrelation + environmental correlation + (1 | seed)
Predictors Fixed Effects Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
Sample number −0.0015 0.7817 0.7817 1 8.3810K 154.0271822 4.6 × 10−35*
Population size 0.0232 1.1290 1.1290 1 8.3810K 222.4711356 1.1 × 10−49*
Migration 0.0904 17.1264 17.1264 1 8.3810K 3374.7537025 0.0*
Selection strength −0.0046 0.0448 0.0448 1 8.3810K 8.8250819 3.0 × 10−3**
Spatial autocorrelation 0.0015 0.0046 0.0046 1 8.3810K 0.9031998 0.34
Environmental correlation 0.0030 0.0194 0.0194 1 8.3810K 3.8301880 0.05
* p < 0.001
** p < 0.01
Tukey test for IBD MAE
pairwise ~ sampstrat
Contrast Estimate SE Z ratio p p
ES - G 0.0499 0.0019 26.3058 0.0 0.000
ES - R 0.0069 0.0019 3.5787 1.0 × 10−3 0.001
G - R −0.0431 0.0019 −22.6633 0.0 0.000
Linear mixed effect model
IBE TPR ~ sample number + sampling strategy + population size + migration + selection strength + spatial autocorrelation + environmental correlation + (1 | seed)
Predictors Fixed Effects Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
Sample number 0.0014 0.0740 0.0740 1 825.0000 6.216035 0.013*
Population size −0.0116 0.0148 0.0148 1 825.0000 1.238654 0.270
Migration −0.0214 0.0164 0.0164 1 825.0000 1.374371 0.240
Selection strength 0.0142 0.0352 0.0352 1 825.0000 2.951667 0.086
Spatial autocorrelation 0.0207 0.0495 0.0495 1 825.0000 4.153425 0.042*
Environmental correlation −0.0032 0.0016 0.0016 1 825.0000 0.130625 0.720
* p < 0.05
Tukey test for IBE TPR
pairwise ~ sampstrat
Contrast Estimate SE t ratio p p
ES - G 0.0133 0.0092 1.4464 0.3176180 0.32
ES - R 0.0098 0.0095 1.0278 0.5595005 0.56
G - R −0.0035 0.0091 −0.3853 0.9214350 0.92
Linear mixed effect model
IBE FDR ~ sample number + sampling strategy + population size + migration + selection strength + spatial autocorrelation + environmental correlation + (1 | seed)
Predictors Fixed Effects Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
Sample number 0.0005 0.0652 0.0652 1 7.5490K 16.4155034 5.1 × 10−5*
Population size 0.0012 0.0029 0.0029 1 7.5490K 0.7388259 0.39
Migration 0.0062 0.0722 0.0722 1 7.5490K 18.1719708 2.0 × 10−5*
Selection strength −0.0012 0.0027 0.0027 1 7.5490K 0.6823366 0.41
Spatial autocorrelation 0.0043 0.0342 0.0342 1 7.5490K 8.6037164 3.4 × 10−3**
Environmental correlation −0.0024 0.0107 0.0107 1 7.5490K 2.6965654 0.10
* p < 0.001
** p < 0.01
Tukey test for IBE FDR
pairwise ~ sampstrat
Contrast Estimate SE Z ratio p p
ES - G 0.0065 0.0018 3.6794 6.8 × 10−4 0.00068
ES - R 0.0012 0.0018 0.6526 0.7909129 0.79000
G - R −0.0053 0.0018 −2.9982 7.7 × 10−3 0.00770
Linear mixed effect model
IBE MAE ~ sample number + sampling strategy + population size + migration + selection strength + spatial autocorrelation + environmental correlation + (1 | seed)
Predictors Fixed Effects Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
Sample number −0.0006 0.1217 0.1217 1 8.3792K 524.2437120 1.4 × 10−112*
Population size −0.0001 0.0000 0.0000 1 8.3792K 0.1295795 0.720
Migration 0.0085 0.1503 0.1503 1 8.3790K 647.3752346 1.3 × 10−137*
Selection strength 0.0007 0.0009 0.0009 1 8.3790K 3.8946914 0.048**
Spatial autocorrelation 0.0026 0.0138 0.0138 1 8.3791K 59.5897497 1.3 × 10−14*
Environmental correlation −0.0018 0.0066 0.0066 1 8.3791K 28.2318946 1.1 × 10−7*
* p < 0.001
** p < 0.05
Tukey test for IBE MAE
pairwise ~ sampstrat
Contrast Estimate SE Z ratio p p
ES - G 0.0023 0.0004 5.6437 5.0 × 10−8 5.0e-08
ES - R −0.0002 0.0004 −0.4591 0.8903506 8.9e-01
G - R −0.0025 0.0004 −6.1009 3.2 × 10−9 3.2e-09

2.2.2 Full plots

2.2.4 Failed fits

Occasionally GDM fails to fit a model, in which case an NA value is assigned. Here we check the proportion of NAs (i.e., cases of failed fit) across the simulations:

Proportion of failed models:

Statistics referenced in-text

Average increase in IBD error from using transect sampling:

## # A tibble: 2 × 3
##   Method `Mean difference` `SD difference`
##   <chr>              <dbl>           <dbl>
## 1 GDM               -0.003           0.045
## 2 MMRR               0.022           0.038

Proportion of times a negative significant coefficient occurs for MMRR:

## # A tibble: 2 × 3
##   sampling   proportion count
##   <chr>           <dbl> <int>
## 1 individual       0.04   550
## 2 site             0.04   345

Comparison of MMRR and GDM

To determine whether MMRR and GDM results were concordant, we visualized the relationship between the coefficients of IBD and IBE between methods under different simulated conditions relevant to performance (i.e., migration and spatial autocorrelation). Overall, there was a strong correlation between the coefficients from the different methods across simulated conditions.

Comparison of average IBE error:

## [1] "MMRR average IBE error: 0.04"
## [1] "GDM average IBE error: 0.01"

Confirmation of appropriate sample size for full models

To confirm that 1000 samples was appropriate for estimating the “full” model coefficients, we plotted the number of samples versus the coefficient to see when the value leveled off. All samples were collected randomly using individual-based sampling. We grouped the IBE coefficients for both environmental variables. Each line represents the result of a simulation, with the thick line representing the mean across simulations. We calculated the mean while grouping by migration and autocorrelation scenario, since these variables had the strongest effects on the value of the coefficients. We also grouped and summarize the GDM coefficients by population size, because population size had a strong effect on GDM coefficients, but not MMRR coefficients. These plots show that coefficient value for both IBD and IBE does not appear to change much with increasing sample size (though a slight decrease in the coefficient values can be observed in GDM). However, variance shrinks substantially with increasing sample size and seems to stabilize after a few hundred samples. Altogether, this supports our use of 1000 samples to represent our full models.

Test of the proportion of times IBE and IBD develops

To confirm that IBD and IBE had time to develop in our simulations, we calculated the proportion of times IBD and IBE were detected as significant by MMRR using the full model. MMRR was used instead of GDM, because GDM was too conservative in detecting IBE. We calculated the proportions for different levels of spatial autocorrelation and migration, since these factors had the potential greatest effect on IBD and IBE. IBD was detected 100% of the time across all scenarios. IBE was detected over ~70% of the time as long as either migration was low or autocorrelation was high. When both autocorrelation was low and migration was high, IBE was detected only 13% of the time; this makes sense since IBE is not expected to develop in scenarios where migration is high and autocorrelation is low.

## # A tibble: 4 × 3
##       m     H `IBD proportion`
##   <dbl> <dbl>            <dbl>
## 1  0.25  0.05                1
## 2  0.25  0.5                 1
## 3  1     0.05                1
## 4  1     0.5                 1
## # A tibble: 4 × 3
##       m     H `IBE proportion`
##   <dbl> <dbl>            <dbl>
## 1  0.25  0.05            0.729
## 2  0.25  0.5             0.960
## 3  1     0.05            0.146
## 4  1     0.5             0.742